The present application is directed to computer vision and more particularly to addressing outstanding issues related thereto. For example, motion segmentation and tracking is a longstanding problem in computer vision. Several approaches have been developed, each of which have their strengths and weaknesses.
Appearance-based approaches to tracking perform dense correlation of a model template against each image frame. Sometimes the correlation is on image pixel values, sometimes on derived features. These methods perform tracking only, not segmentation and discovery of moving objects. They need to be initialized with the target object either manually or by some other algorithm. An example of such a concept is discussed in Zdenek Kalal, K. Mikolajaczyk, and J. Matas, “Tracking-Learning-Detection,” Pattern Analysis and Machine Intelligence, 2011.
Feature tracking/optical flow methods perform object segmentation (moving object detection and initialization) in addition to object tracking. These methods first find keypoint features in an incoming image frame. These features are tracked from frame to frame by local correlation or optical flow. Thus, feature correspondences are known for a pair or sequence of frames. Coherently moving sets of features are found by a number of means including Random Sample Consensus (RANSAC), linear subspace projection, or region growing. These methods sometimes use keyframes to establish feature configuration at one point in time, then segment moving objects according to deviation from keypoints' locations in the keyframe. One group in particular, led by Stan Birchfield of Clemson University, has developed motion segmentation and tracking algorithm that has been applied to traffic scenes. This is the basis of a commercial product by a company called TrafficVision. The Birchfield approach requires relatively smooth and slow motion, and does not re-detect known objects in each frame which is necessary for tracking under severe camera jerkiness, e.g., S. J. Pundlik and S. T. Birchfield, “Motion Segmentation at Any Speed,” Proceedings of the British Machine Vision Conference (BMVC), Edinburgh, Scotland, pages 427-436, September 2006.
There is a great deal of academic work on dense layered optical flow estimation which delivers object segmentation. These methods are computationally expensive and in many cases require evaluating an entire image sequence as a batch. Therefore it is not suitable for real-time use on currently available standard computing hardware.